Attention for Computer Graphics Rendering

We present a method to accelerate global illumination computation in pre-rendered animations by taking advantage of visual saliency and limitations of the human visual system. A visual importance map, constructed from saliency and visual sensitivity model, is used to accelerate rendering. Results indicate an order of magnitude improvement in computational speed. I) Efficient Realistic Rendering of Synthetic Images Accurate lighting computation is one of the key elements to realism in rendered images. In the past two decades, many physically based global illumination algorithms have been developed for accurately computing the light. These algorithms simulate the propagation of light in a three-dimensional environment and compute the distribution of light to desired accuracy. Unfortunately, the computation times of these algorithms are high. For a reasonably complex scene with dynamic objects, physically accurate global illumination computation for each image frame can take many hours. Thus one of the main research goals in field of rendering is to develop algorithms for faster and realistic rendering. In this chapter, we describe one such algorithm (Yee et al. 2001) in which we improve the speed of rendering computation by allocating computational effort to various parts of a scene in proportion with the visual saliency of the areas of the scene. The basic idea behind this work is that only a small fraction of a viewed scene is visually important. Thus we save significantly on time by terminating the computation early in areas of the image that are not visually important. II) Visual Attention Based Rendering Algorithm The first step in our algorithm is to derive the saliency of the areas of the scene visible in the current view. To be able to do this we first obtain a quick estimate of the final image by hardware rendering a preview image of the scene. The preview image contains important visual cues such as the color of objects, the textures on the objects and approximate contrasts. We apply a model of visual attention to the preview image to obtain a saliency map (see chapter “Models of Bottom-Up Attention and Saliency”). The next step of our algorithm is to apply a model of human visual system to the preview image to obtain visual sensitivity map for the viewed scene (Ramasubramanian et al, 1999). The model of the human visual system takes into account the loss of contrast sensitivity in the presence of high spatial frequencies and motion. We modulate the saliency map with the visual sensitivity map to obtain a map of visual importance. We use this map of visual importance to guide our global illumination computation. We allocate less computational effort to areas of less importance and more to areas of higher importance. Figure 1(a) shows an image computed using our efficient global illumination algorithm. Figure 1(b) is the importance map for the image, generated using visual saliency and the model of the human visual system. The brighter pixels correspond to low visual importance and received less computational effort. Similarly, darker pixels correspond to high visual importance and hence received more computational effort. For example, the sofa near the center received more computer time. The rational behind this is: the sofa stands out in the corner of the scene as salient, and it has very little high frequency content to mask the error, hence correspondingly more computation effort is needed to ensure that the result looks accurate to the visual system. Figure 1(a). Image drawn by Global Illumination. Figure 1(b). Visual Importance derived from Visual salience and Contrast Sensitivity. Bright areas are less important. By using a model of visual attention to guide the global illumination algorithm, we found that we can speed up calculation by an order of magnitude. Figure 2 below shows the distribution of the computational effort. A bright dot in the image corresponds to a unit amount of effort spent by our global illumination algorithm. The density of the white dots is proportional to the computational effort spent by the algorithm. On the right, we have the effort distribution by our algorithm. For comparison, on the left we have the effort distribution for a reference solution generated by switching off the attention based acceleration. As can be seen, our algorithm used far less processing power to compute the lighting with equal visual accuracy. Furthermore, computation was concentrated in visually important regions such as the sofa, the flower petals and the bouncing spheres. Figure 2. Sampling difference between the reference and the importance guided solutions. III) Related Work In The Field We have come across Haber et al’s (2001) work that makes attempts similar to our work by using visual attention to speed up high quality rendering computation. It uses the saliency map to enhance interactive graphics by performing ray tracing in areas that are deemed to be visually important. This allows them to simulate computationally expensive effects such as glossy surfaces at interactive rates. ReferencesHaber, J., Myszkowski, K., Yamauchi, H., and Seidel, H. (2001). “Perceptually guided correctivesplatting”, Computer Graphics Forum, Eurographics Proceedings Issue, Vol 20(3), 2001.Ramasubramanian, M., Pattanaik, S. N., and Greenberg, D. P., "A Perceptually Based PhysicalError Metric for Realistic Image Synthesis", Proceedings of SIGGRAPH’99, pp. 73-82, LosAngeles, 8-13 August 1999.Yee, H., Pattanaik, S. N., and Greenberg, D. P. (2001). “Spatio-temporal sensitivity and visualattention for efficient rendering of dynamic environments”, ACM Transactions on Graphics,20(1), pp. 39-65, January 2001.